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DETAILED ACTION 
Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the 
United States before the invention thereof by the applicant for patent, or on an international application 
by another who has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this 
title before the invention thereof by the applicant for patent. 

2. Claims 1-2, 4-5,7-20 and 23, are rejected under 35 U.S.C. 102(e) as being 
anticipated by Harakawa et al (U.S. patent 6,385,331 )). 

Regarding claim 1, Harakawa discloses an operation recognition system, 
comprising: an object to be operated; at least one camera configured to acquire image 
data of an operator, at least a predetermined number of image data being. acquired in a 
predetermined time period (column 1, lines 5-10, the present invention relates to a hand 
pointing, and more specifically to a hand pointing apparatus for picking up a person to 
be recognized and for determining a position or a direction pointed to by the person to 
be recognized, also column 18, lines 6-16, an image of the information input space is 
picked up by the video camera and the image is then output from the video camera, 
whether a predetermined time period has passed since the illuminator was switched on 
is determined); 

a processor configured to perform recognition processing on the acquired image 
data, so as to define a first virtual plane in between the object and the operator, and to 
determine if a predetermined part of the operator penetrates the first virtual plane 
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(column 20, lines 44-56, the position (plane coordinates) of the information inputting 

4 

person on the floor surface is set to the plane coordinate, also column 22, lines 38-50, 
in step 176, based on the three -dimensional coordinates of the reference point P0 of 
the information inputting person calculated in the previous step, and the three- 
dimensional coordinated of the feature point Px calculated in step 174, the direction of 
an extended virtual line (see Fig. 1 1 ) connecting the reference point and feature point is 
determined as the direction pointed to by the information inputting person, and virtual 
line are calculated in order to determine the position pointed to by the information 
inputting person); 

and a controller configured to control the object based on the determination (Fig. 
6, column 17, line 60 through column 18, line 16, control processing which is regularly 
carried out by the controller 22, an image of the information input space is picked up by 
the video camera based on determination). 

Regarding claim 2, Harakawa discloses the system according to claim 1, wherein 
the processor is configured to recognize a predetermined motion of the operator and the 
controller is operative when the processor recognizes the predetermined motion of the 
operator (column 18, line 6-29, control processing which is regularly carried out by the 
controller 22, an image of the information input space is picked up by the video camera 
based on determination, also column 20, line 57 through column 21, line 3). 

Regarding claim 4, Harakawa discloses the system according to claim 1 , wherein 
the processor is configured to recognize an operational authority of the operator based 
on the recognition processing (column 2, line 59 through column 3, line 15, by 



Application/Control Number: 1 0/784,1 64 Page 4 

Art Unit: 2624 

calculating a three-dimensional coordinate of feature point of the person to be 
recognized a point whose position is changed in response to the motion by person to be 
recognized, further column 24, lines 12-23, refer to information inputting person). 

Regarding claim 5, Harakawa discloses the system according to claim 1, wherein 
the processor is further configured to detect a part of the first virtual plane penetrated by 
the predetermined part of the operator, and wherein the controller controls the object 
based on the detection (column 10, lines 12-37, executed a predetermined processing 
when a change is detected by the first detecting). 

Regarding claim 7, Harakawa discloses the according to claim 1, wherein, when 
the processor is configured to recognize a length of an operator's arm, and to define the 
first virtual plane at a position within the recognized length from the operator (column 
10, lines 12-20, refer to hand, the arm or the like). 

Regarding claim 8, Harakawa discloses the system according to claim 1, wherein 
the first virtual plane is defined at a variable position based on the recognition (column 
20, lines 44-56, the position (plane coordinates) of the information inputting person on 
the floor surface is set to the plane coordinate, also column 22, lines 38-50, in step 176, 
based on the three -dimensional coordinates of the reference point P0 of the 
information inputting person calculated in the previous step, and the three-dimensional 
coordinated of the feature point Px calculated in step 174, the direction of an extended 
virtual line (see Fig. 1 1 ) connecting the reference point and feature point is determined 
as the direction pointed to by the information inputting person, and virtual line are 
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calculated in order to determine the position pointed to by the information inputting 
person). 

Regarding claim 9, Harakawa discloses the system according to claim 1 , wherein 
the processor is configured to determine a penetration when the predetermined part of 
the operator penetrates the first virtual plane by more then a predetermined length 
(column 10, lines 12-29, refer to predetermined length). 

Regarding claim 1 1 , Harakawa discloses the system according to claim 1 , 
wherein the processor is configured to determine a penetration when the predetermined 
part of the operator penetrates the first virtual plane for more then a predetermined 
period (Fig. 15, column 25, lines 32-47). 

Regarding claim 13, Harakawa discloses the system according to claim 1, 
wherein the controller is configured to control the object based on a length by and a 
time period in which the predetermined part of the operator penetrates the first virtual 
plane (column 18, line 6-29, control processing which is regularly carried out by the 
controller 22, an image of the information input space is picked up by the video camera 
based on determination, also column 10, lines 12-20, refer to hand, the arm or the 
change in the length). 

Regarding claim 14, Harakawa discloses the system according to claim 1 , 
wherein the object includes a display configured to display one or more operation items, 
and wherein the controller is configured to control the implementation of one of the 
operation items based on the recognition and the determination (column 15, lines 17-40, 
displaying various types of information). 
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Regarding claim 15, Harakawa discloses the system according to claim 14, 
wherein the processor is configured to recognize a predetermined motion of the 
operator, and the processor is configured to determine that the predetermined motion 
represents a click operation (column 30, lines 35-49, click motion). 

Regarding claim 17, Harakawa discloses the system according to claim 13, 
wherein the processor is configured to recognize a predetermined motion of the 
operator, and the processor is configured to determine that the predetermined motion 
represents a drag operation (Fig. 22A and 22B, column 30, lines 35-49, bending and 
turning and extending the wrist or finger (dragging) and forward or backward motion of 
the hand). 

Regarding claim 18, Harakawa discloses the system according to claim 1, further 
comprising a second object configured to operate in response to the object (column 33, 
lines 13-30, response to the result of the determination). 

Regarding claim 19, Harakawa discloses the system according to claim 1 , 
wherein the processor is further configured to define a virtual keyboard in the first virtual 
plane, and wherein the controller is configured to control the object in relation to a 
keyboard input via the virtual keyboard based on the determination (see claim 1 , also 
column 15, lines 32-41, refer to keyboard). 

Regarding claim 23, Harakawa discloses the system according to claim 1, 
wherein the object includes a display configured to display one or more operation items, 
the display including a projector and a screen, wherein the first virtual plane is defined in 
between the screen and the operator (column 15, lines 8-30, operator and screen). 



Application/Control Number: 10/784,164 Page 7 

Art Unit: 2624 

With regard to claims 10, 12, 16 the arguments analogous to those presented 
above for claims 1, 9, 1 1, and 15 are respectively applicable to claims 10, 12, 16. 

With regard to claims 20 the arguments analogous to those presented above for 
claims 1 , 19 are respectively applicable to claims 20. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 3 and 24-35, are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Harakawa et al (U.S. patent 6,385,331 ) in view of Rubbert et al (U.S. patent 
7,029,275). 

However regarding claim 3, Harakawa does not explicitly state its corresponding 
" recognize predetermined voice". On the other hand Rubbert teaches (column 21, lines 
37-54, the user interface also includes a keyboard and mouse for manipulating the 
virtual model of object. The base station may alternatively include a voice recognition 
module that is trained to recognize a voice command). 

Therefore it would have been obvious to one having ordinary skill in the art at the 
time the invention was made to modify Harakawa invention according to the teaching of 
Rubbert because combination of Harakawa and Rubbert provides computerized, 
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interactive method and associated system for virtual model and voice recognition, which 
can easily implemented in an scanning device. 

Regarding claim 27, Harakawa discloses the system according to claim 1 , 
wherein the object includes an appliance (column 10, lines 12-29, the plurality of images 
and for detecting a changes in either the area). 

Regarding claim 30, Harakawa discloses the system according to claim 1 , 
wherein the object includes a display configured to display a virtual plane position 
change window and a position of the first virtual plane is changed in accordance with 
the determination (see claim 1, also column 10, lines 12-29, the plurality of images and 
for detecting a changes in either the area). 

Regarding claim 32, Harakawa discloses the apparatus according to claim 31 , 
further comprising a shape detector configured to detect a shape of the operator's hand; 
and a recognition mechanism configured to recognize that the operator has an 
operation authority when the detected shape is identical to a predetermined shape 
(column 20, line 57 through column 21, line 12, based on the shapes of image). 

With regard to claims 24-26, 28 and 29 the arguments analogous to those 
presented above for claims 1 , 3, 27 and 30 are respectively applicable to claims 24-26, 
28 and 29. 

With regard to claims 31 and 33-35 the arguments analogous to those presented 
above for claims 1, 3, 5 and 7 are respectively applicable to claims 31 and 33-35. 
5. Claim 6, are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Harakawa et al (U.S. patent 6,385,331) in view of Tanide et al (U.S. patent 6,201,516). 
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However regarding claim 6, Harakawa does not explicitly state its corresponding 
"recognize an eye of the operator as a view point, and to define the first virtual plane 
between the object and the view point". On the other hand Tanide in the same field of 
virtual reality teaches (column 2, line 65 through column 3, line 7, the visually 
recognized range of a viewer (eye), a projecting system giving continuity in an image 
and a high feeling of presence). 

Therefore it would have been obvious to one having ordinary skill in the art at the 
time the invention was made to modify Harakawa invention according to the teaching of 
Tanide because it provides system for sensation is defined as a system giving a virtual 
experience and desire motion of an image operated interactively by a viewer. 

Allowable Subject Matter 

6. Claims 21 and 22 objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

Other prior art cited 

7. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

(U.S. patent 7,161 ,616) to Okamoto et al is cited for image processing device 
and monitoring system. 

(U.S. patent 7,007,236) to Dempski et al is cited for lab window collaboration. 



Application/Control Number: 10/784,164 



Page 10 



Art Unit: 2624 

(U.S. patent 6,842,175) to Schmalstieg et al is cited for tools for interacting with 
virtual environments. 

(U.S. patent 6,407,762) to Leavy is cited for camera-based interface to a virtual 
reality application. 



8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Seyed Azarian whose telephone number is (571 ) 272- 
7443. The examiner can normally be reached on Monday through Thursday from 6:00 
a.m. to 7:30 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella, can be reached at (571) 272-7778. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application information Retrieval (PAIR) system. Status information for published 
application may be obtained from either Private PAIR or Public PAIR. 
Status information about the PAIR system, see http:// pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). 

Seyed Azarian 
Patent Examiner 
Group Art Unit 2624 n / 
April 8, 2007 J MH/ rf^P&f* 
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